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ABSTRACT 


The  paper  1  by  Dantzig  and  Wolfe  suggested  the 
need  for  developing  new  techniques  for  solving  linear 
programming  problems  with  a  special  matrix  structure. 
A  number  of  techniques  have  appeared  since  then.  In 
this  report,  an  algorithm  for  solving  a  structured 
linear  programming  problem  with  a  very  large  number 
of  blocks  is  given.  The  main  feature  of  the  method 
as  described  in  3  is  to  carry  out  the  computation 
with  the  help  of  a  smaller  basis  whose  order  is  e- 
qual  to  the  number  of  linking  equations  coupling 
together  the  various  blocks. 


J 


An  Extension  of  Generalized  Upper  Bounded  Techniques 
for  Linear  Programming 


1.  Introduction* 

The  decomposition  principle  of  Dantzig  and  Wolfe  [1]  for  solving  a  sys¬ 
tem  with  a  block  diagonal  structure  is  well  known.  An  efficient  computational 
procedure  has  been  given  by  Dantzig  and  Van  Slyke  [2]  [3]  for  solving  a  very 
large  block  diagonal  structure  where  each  of  the  blocks  contains  just  one  e- 
quation.  Allowing  for  the  possibility  of  several  equations  in  each  block,  it 
would  be  interesting  to  investigate  the  corresponding  technique  for  a  large 
number  of  blocks.  The  purpose  of  this  paper  is  to  describe  this  technique. 

The  main  feature  of  this  method  as  described  in  [2]  [ 3 J  is  to  carry  out  the 
computation  with  th^*  help  of  a  smaller  basis  whose  order  is  equal  to  the  num¬ 
ber  of  linking  equations,  coupling  together  the  various  blocks. 

We  shall  concern  ourselves  with  the  problem  of  solving  the  following 
structured  system 


Vo  +  Vi+ 


Blxl 


Vl 


xi  1  0 


+ 


The  author  is  thankful  to  Professor  R.  M.  Van  Slyke  and  Mr.  Paul  Rech  for 
their  comments. 


where  the  first  component  of  the  vector  Xg  ,  unrestricted  in  sign,  is  to  be 
maximized. 

The  system  in  the  expanded  form  is  given  in  Figure  I. 

A  problem  which  may  have  this  structure  is  a  multiple  plant  model  where 
each  plant  is  represented  by  a  different  block,  the  blocks  being  coupled  together 
by  raw  material  allocation  and  product  distribution  [4]. 

We  start  with  some  definitions  and  theorems  in  the  next  section,  and  in 
the  following  the  algorithm  will  be  described.  We  conclude  with  a  numerical 
example  illustrating  the  application  of  the  algorithm. 
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Figure  I 
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2.  Definitions  and  Notations 


We  shall  refer  to  the  system  of  equations  in  (1)  which  couple  together 
the  different  blocks  as  the  "linking  set".  The  k-th  set  of  columns  or  vari¬ 
ables  is  a  set  of  columns  or  variables  that  a  linking  set  has  in  common 

with  the  k-th  block.  The  set  of  columns  does  not  have  any  block  down 

s  t 

below  and  therefore  has  all  the  zero  entries  in  (Mfl)  through  rows. 

We  assume  that  the  whole  system  (1)  and  the  different  blocks  are  all 
of  full  rank.  The  basis  for  the  system  (1)  is  of  the  form 

J1  J2  \ 

where  underscoring  is  used  in  designating  the  components  for  the  full  system 
so  as  to  distinguish  them  from  the  first  M  components  which  are  represented 
without  underscoring.  For  convenience  of  notation,  we  assume  that  the  number 
of  equations  in  block  k  is  m^  (k  ■  1, . . . ,L)  . 


Theorem  1:  Every  basis  of  (1)  rnust  have  at  least  columns  from  *he  set  0  . 

Proof .  Let 

J 1  J  2 

Bf  "  (A  ,  A  ) 

be  the  basis  for  the  full  system.  Now  consider  any  other  M^-  vector 


(0,  J, 


* ,0,bM  +l,bM  +2 
P  P 


b  ,0, ...  ,0, ....  ,0)^  ,  0  <_  p  <_  L-l 
P+l 


there  exist 


such  that 
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/ 


l  =  (0,...,0,b^  +1’^M  +2  *  *  *  *  *^M  »0»**»»0) 

i  =  l  P  P  p+1 


I .  e 


'A 


k  -  M  +1,  M  +2 . M  , 

P  P  P+1 


0  <  p  <  L-l 


Ji 

wh  i  cl',  shows  that  there  are  m  or  m°re  A  in  the  (p+l)th  block  sine*'  each 
of  the  blocks  is  assumed  to  be  ot  the  full  rank.  This  proves  the  theorem. 


Theorem  2:  The  number  of  sets  S^(i^O)  having  basic  variables  more  than  the 

number  of  equations  m ^  in  the  corresponding  block  cannot  exceed  M-l. 

Ft cof .  The  system  (1)  is  or  fall  rank;  therefore,  the  basis  consists  of 

r 

M  )  m  vectors.  But  it  has  been  shown  in  Theorem  1  that  each  set  S  con- 
i-*l  1  1 

tains  at  least  m^  vectors,  eliminating  all  the  basic  vectors  except  for  M  of 

them.  One  of  these  is  the  variable  to  be  optimized;  then  at  most,  M-.l 

basic  variables  remain  to  itu  a  up  sets  (i  f  0)  with  more  than  m^  basic 

var  rabies . 

Borrowing  the  terminology  from  [2],  we  call  the  set  Sq  and  all  those 
secs  having  basic  variables  more  than  the  number  of  equations  in  the  cor¬ 

responding  block  as  essential  sets.  All  other  sets  are  called  inessential  sets. 


3.  Discussion  of  the  Algorithm 

Let  us  suppose  that  we  have  at  hand  an  initial  basic  feasible  solution  to 
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(1)  which  can  always  be  obtained  by  using  the  Phase  I  procedure  of  the  simplex 
method.  This  gives  a  division  of  the  sets  (i  i*  0)  into  essential  and  ines¬ 
sential  groups  as  explained  above.  All  the  subproblems  *  b*  that  belong 

to  the  inessential  sets  are  solved  independently  by  setting  to  zero  the  variables 
not  associated  with  the  basis.  Since  the  essential  set  contains  more  ba¬ 

sic  variables  than  the  number  of  equations  m^  in  the  block  associated  with  that 
set,  we  may  regard  the  independent  m  columns  of  the  basis  of  (1)  (the  existence 
being  implied  by  Theorem  1)  in  such  a  set  St  as  key  colrmns  and  the  corres- 

k-i 

ponding  variables  as  key  variables.  The  notation  A  x  and  x.  will  be  used  for 

kl 

key  column  and  key  variable  respectively.  For  the  sake  of  uniformity,  let  us 
call  all  the  columns  of  the  basis  associated  with  inessential  sets  as  key  columns, 
having  solved  the  subproblems  corresponding  to  the  inessential  sets  for  key 
variables  as  stated  above,  we  next  solve  the  subproblems  in  the  essential  sets 
for  the  key  variables  by  setting  the  remaining  variables  in  that  set  to  zero. 

Let 
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be  the  coefficient  matrix  of  the  key  variables  in  the  set  S  .  We  note  that 
(f )  ■■( t ) 

the  matrices  XJ  ,  v  correspond  to  the  key  variables  in  the  '  linking  set" 
and  the  t-th  block  respectively. 


For  every  non-key  column  AJeS^ 
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where 


•4/ 


-  -  (3) 


The  superscript  within  parentheses  is  used  to  indicate  the  number  of  the  block. 
St  1 

Since  the  (M+l)  through  components  of  P_J  are  zero,  we  may  also  write 

PJ  -  AJ  -  tfC)  X(t)  . (4) 

Repeating  the  above  procedure  for  every  non-key  columns  and  transferring 
the  key  columns  to  the  right,  we  obtain  the  reduced  system 


lYjPj  -  Q  . (5) 

where 

«  '  b  -  l  ^Vr)  •  . <6> 

r 


here,  ^(r)  *  t*ie  vector  °f  key  variables  in  r-th  block,  is  determined  by 

solving  the  subproblem  in  that  block  on  setting  non-key  variables  to  zero. 

We  shall  now  show  that  the  basis  B  for  the  reduced  system  (5)  consists 

of  the  column  associated  with  the  variable  Xq  and  all  the  non-key  columns 

left  in  after  these  are  modified  as  in  (2). 

f  '* 

Assume,  therefore,  that  after  performing  the  necessary  operations  and 

^1  *-2 

transfer  as  indicated  above,  we  are  left  with  the  columns  p  ,  p  ,...,p 

Wt  assent  that  these  form  the  basis  B  for  the  system  (5).  If  not,  there  exist 
not  all  zero  such  that 
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* 


*k 

}  ,  F  =  0 


t .  o 


M 

\  „  k 


t,  P 

k— 


=  U 


K  =] 


wii.;u,  jp  using  (2)  yields 


I  6jA  -  0 


i=l 


J1  .J2 


Lx 


where  nut  ill  ek  are  zero.  But  this  implies  that  (A  t . . . , A  ")  are 

dependent  and  hence  a  contradiction. 

In  order  to  generate  a  better  solution  or  test  optimality,  we  com¬ 
pute  the  price  vector 


H  >  .2) 

i  •  a  *  ' 


.( L) 


C!T  ■»  W  » IA  >  •  •  •  J 


(1)  (1)  (1) 


(L) 


u  t 


M+l  ’  M+2 1 


Mj  . M, 


ditc-e  ,uu  determined  such  that 


and 


® 1  ^  ;|JL( " J ; . . . ;  ^  L) )  AJ 1  -  0  i  -  2,...,M( 


J 1  0 
(A  «  A  ) 


L«t  rr  be  tee  first  row  of  the  inverse  of  the  basis  B  for  the  reduced  syste: 
i  i .  t  Ik  n 


*  0 

B  P  =  1 


<p  1  -  r°) 
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and 


*  4k 


I?  *0  . 


k  -  2, . . .  ,M 


*  *(t) 

Hence,  is  a  set  of  prices  for  (5).  Now  to  compute  ,  (t  ■  1,...,L)  , 

we  observe  that  for  the  set  S 


«v(1V2);-.  v(L)a(t>  -  ° 


i.e. , 


where 


-j fv(tVt)'1  -  -nW 


(t  *  !,•••» L) 


*  *(i)  *(2)  *(l) 

We  now  claim  that  (||  J|JL  )  are  a  set  of  prices  for  the  whole 

*  *(i)  *(L) 

system  (1).  For  in  the  manner  we  obtained  H.  ;|A  ;|A  it  is  obvious  that 


ji  ji 

if  _A  tS  or  A  is  a  key  column,  then 


«V(1,;|r(2>s...;|»*<t))A1-i 


®*;(i  (1>;P*(2);...;|**a,)AJl  -  0  (1  **  1) 


J  i 

On  the  other  hand,  if  A  is  not  a  key  column  and  A  ,  then 


- (8) 


wnere  t 


is  the  part  of  the  column  A  in  til*  t-*th  block. 
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rum  (5),  (4),  (7)  and  (8)  we  obtain 


J. 


-B*(* 


/AJi  -u(t)A<0 


j 


) 


* 

*U  P  for  some  k 


0  . 


k  *  i  J  )  *  (  1  ■)  *  ( L ) 

rhuh  ;|Ji  ;|A  )  is  a  pricing  vector  for  the  system.  Now, 

pricing  out  the  non-basic  columns  we  enter  the  column  fe  ■  wnich 


es(t)  -  «Va,:---*<LV(t) 

-  Min  U*i(**<1>;...;|/<L))AJ<t) 

assumes  minimum  for  t  *  0,1,,.., L  and  this  minimum  is  negati  e.  If  the 

minimum  turns  out  to  be  non-negative,  we  terminate  and  are  at  the  optimal  sole- 

s 

tion.  Assume  this  is  not  the  case  and  the  column  A  c.S  qualifies  for  entry 
into  the  basis,  the  next  step  is  to  find  out  the  vector  that  drops  out  of  the 

5 

basis.  For  this  purpose  we  express  the  column  A  and  the  vector  b  in  terms 
of  the  current  basis. 

— g 

II  P  denotes  the  representation  of  P  in  terms  of  the  basis  b  ,  then 


p’.lFOft1 

^  i 


n.  (t  )  (t  ) 

i  tt  i  x  i 


=  (A  1  -U  1  *  1  ) 

L  i 


wiien  P  1  S  and  the  i-tli  basic  variable  P  of  b  is  in  column  number 

t  l 

i 


I  iuM  flint  , 
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lienee,  if 


n,  _(t  J  +^l)  Jl) 


A-  -  I  pT  (A  1  --O'  4'X- V>+1T'V 
^  i  n. 


then 


,S  -  XU) 
li  Ak 


-i 


n/s 


.s  v8  a^ 
A  ■  1  qA  A 


xu)  p8 
Ak  .  Pi 
n.  n/n. 

A  eS„ 


ji  kn(£) 
if  A  -  A 


-  I 


A(t)  ?S 

Ak  ,  1  i 

n.  n/n, 

A  eS 


Ji  k  (t) 

if  A  «  A  ,  t  i  1 


Ji  nt 

if  A  "A  for  sore  t 
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otherwise 


A  similar  consideration  shows  that 


_  i, 

Q  -  I  Q,P  1 
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I  VA 


<t.)  <t.) 

-u  1  *  1  ) 


and  therefore,  using  (b) 


(tj  (tj 


lQt(A  l  -fl 

i  i  c 
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i  '  ’i  S  S  1 1 UW  Ki  t  llut  If 


1.  ij in: 
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*  ^  i 
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-  I 


As, 
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Vk 
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if  AJi  =  /n(° 


=  Q. 


Ji  \ 

ii  A  “A  tor 
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otherwise . 


—  *  Min 


qS>0 
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qi 


i  *  2  .... K 
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jr 

the  i  tne  simplex  method  requires  that  A  be  dropped  from  the  basis.  We 
ok  ijr-i  t.  in'  following  cases. 


Jr  S 

,  i_-j  .  l :  _A  and  .A  belong  to  the  same  set  which  is  inessential 

jr 

tun  A  it-piuces  A  us  key  column  and  this  does  not  introduce  any  cti  .n 
m  Me  s  oliei  oasis  B  . 


it-L  is  lenote  bv  Q  the  new  value  of  Q  ,  then 


Q  =  a 


-l 


k  (£) 

b  -  lU(l)Xk(i)  +  A  3  \ 

i  KU)  (£) 


'Sx 

i  \  A 


—  -I  s  .ka(£)  .  ka(0  Jr 

-  Q  -  i.  (o  x  -  M  x  )  ,  A  =  A 

a(H) 


We  now  Jetevnine  kite  new  set  of  prices  and  the  column  eligible  for  entry 


some 


now 
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Case  2,  Assume 


Jr 

A  eS 


m 


and  is  not  a  key  column  and  set  P 


J‘  AJr  -iAki(m) 
i 


Vj 


The  updating  of  B  in  this  case  is  accomplished  by  pivoting  on  the  element 

s  J, 

that  lies  in  the  P  column  and  in  the  row  of  P  ^  .  We  observe  that 


P  -  k.P  1  +...+k  P  r  +...+k,Pl. 
1  r  h 


j  4 

r  r 

where  the  column  A  is  the  column  P  in  the  smaller  basis. 
*-l  -1 

Hence,  B  *  E  B  where 


-k 


1/k 


-k 


2/k 


1/k 


-k 


M/k 


Q  is  updated  by  premultiplication  with  the  matrix  L 

Case  3  If  A.  cS^  and  is  a  key  column,  then  in  the  set  we  make  some 

r  jr 

column  of  the  basis  3  ,  say,  A  ,  key  in  place  of  A  .  The  existence  of 
at  least  one  such  column  is  implied  by  Theorem  1.  The  columns  of  B  in  the  set 


S  have  the  form: 
h 
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Before  A _ is  dropped 


we  find  that  B  p 


•  •  • 


f  •  •  •  » 


l 

B  * 


l 


Therefore,  the  relation  between  the  old  basis  B  and  the  new  basis  B  ob¬ 
tained  in  changing  the  key  variable  can  be  expressed  as 

B  -  BM 

where 
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Hence,  changing  the  key  column  amounts  to  premultiplying  the  old  B  by  M 
to  obtain  thi  new  B  ^  . 

^  r  '-1 

Now  A  is  a  non-key  column  in  the  set  ,  and  updating  of  B  and 

Q  are  accomplished  as  in  Case  2.  Therefore,  we  are  ready  for  the  next  iteration. 

We  next  consider  the  following  example  illustrating  the  above  procedure: 
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Example: 


Max  x. 


Subject  to: 


4xn  +  x.  +  2x„  -  x_  +  x.  -  lOx,.  +  3x,  -  4x  +  2xQ  +  Ox  +  2x_n  =  12 

U  1  4  3  4  3  b  /  oy  ID 

0x„  +  7x.  +  x„  +  4x_  -  3x.  +  18X,.  -  6x^  +  x_  -  x,,  +  x.  -  xln  =  2 

U  1  4  J  4  30  /  6  y  1U 

2xn  -  x,  +  2x0  -  x_  +  x.  -  2x_  +  Ox,  -  3x7  +  2x  -  x  -  Ox,  =  7 

01234  56  789  1.0 


3x2  -  x3  +  x^  -  Ox^ 


-4x„  +  x.  +  85-  +  16xc 

2  3  4  5 


20 


Ox,  +  3x-, 

6  7 


x8+  x9 


-  Ox 


10 


3x0  -  4x?  +  xg  +  Ox,  +  Ox1Q  -  2 


6x^  -  10x^  +  3xg  +  2xg  -  2x1q  *  7 


xt  >  0  (  i  +  0  ) 


Assume  that  we  have  an  initial  basic  feasible  solution  and  the  columns  A 


0 


A^,  A3,  A**  A*5,  pJ  ,  A^,  A^  are  in  the  basis. 

2  3  6  7  8 

Take  A  ,  A  key  in  the  set  ,  and  A  ,  A  ,  A  key  in  the  set  S0 

n  .  /  .0  .4  _  .(1)  2  _  .(1)  .3  9  _  ,(2)  6  _  .(2)  7  ,  (2)  8  . 

B  (A  »  A  ^  ]./  U  A  ^2/4  ^  ^  A 1/9  A  ^2/9  ^  ^  3  /  9  ^ 


where 
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.ience 


A 

(tr ; 


(J) 


*(2)  . 

|A  ) 


(1/2,  0,  -1/2,  J,  0,  -1/2,  -1/2,  U) 
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where 

[x2,x3]  =  [-25,  -80]  ,  [x&  ,  x?,  xg]  «  [7/3,  4,  11] 


are  obtained  on  solving  the  subproblems  separately. 
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Pricing  out  the  non-basic  columns,  we  find  that  the  vector  A  qualifies 
for  entry  into  the  basis. 

Now 
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We  again  compute  the  pricing  vector  and  find 
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All  the  columns  price  out  optimally,  and  hence  we  have  the  optimal  solution. 
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Thus 
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from  which  the  optimal  solution  reads  as  follows: 

xQ  *  2737/1146,  xl  =  117/382,  x2  =  548/373,  x4  z  407/191, 

x^  =  971/2292,  x^  =  5/9,  xQ  *  1/3,  x^  =  4/3,  r  x?  =  x^  =  0  . 
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